Medical image processing device and medical observation system

ABSTRACT

A medical image processing device includes: a first captured image acquisition unit configured to acquire a first left-eye image and a first right-eye image; a second captured image acquisition unit configured to acquire a second left-eye image and a second right-eye image; a superimposed image generation unit configured to superimpose corresponding pixels of the first left-eye image and the second left-eye image on each other to generate a left-eye fluorescence superimposed image and superimpose corresponding pixels of the first right-eye image and the second right-eye image on each other to generate a right-eye fluorescence superimposed image; and a display controller configured to generate a display image from the first left-eye and right-eye images, the second left-eye and right-eye images, the left-eye and the right-eye fluorescence superimposed images, wherein the display image includes a stereoscopically observable three-dimensional fluorescence superimposed mage generated from the left-eye and the right-eye fluorescence superimposed images.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority from Japanese Application No.2020-048448, filed on Mar. 18, 2020, the contents of which areincorporated by reference herein in its entirety.

BACKGROUND

The present disclosure relates to a medical image processing device anda medical observation system.

In the related art, there is known a medical observation system thatadministers a fluorescent substance such as indocyanine green into aliving body and irradiates an observation target with excitation lightthat excites the fluorescent substance to fluorescently observe a lesionin which the fluorescent substance is accumulated (see, for example, JP2015-29841 A).

In the medical observation system described in JP 2015-29841 A, thefollowing first and second captured images are acquired andcorresponding pixels of the first and second captured images aresuperimposed on each other to generate a fluorescence superimposedimage.

The first captured image is an image obtained by irradiating asobservation target with normal light, which is white light, andcapturing the normal light reflected by the observation target with animage sensor.

The second captured image is an image obtained by irradiating anobservation target with excitation light that excites a fluorescentsubstance such as indocyanine green and capturing the fluorescence fromthe observation target excited by the excitation light with ahigh-sensitivity image sensor.

Further, a doctor excises or sutures a lesion while confirming thefluorescence superimposed image.

SUMMARY

The fluorescence superimposed image displayed by the medical observationsystem described in JP 2015-29841 A is a two-dimensional image. On theother hand, a three-dimensional image that may be observedstereoscopically has a sense of depth, and thus, the image makesexcising or suturing of the lesion easy.

Therefore, there is a demand for a technique capable of improvingconvenience by generating a three-dimensional fluorescence superimposedimage that may be observed stereoscopically.

According to one aspect of the present disclosure, there is provided amedical image processing device including: a first captured imageacquisition unit configured to acquire a first left-eye image and afirst right-eye image having parallax each of which is obtained bycapturing light from an observation target irradiated with light in afirst wavelength band, the observation target emitting fluorescence whenirradiated with excitation light in a second wavelength band differentfrom the first wavelength band; a second captured image acquisition unitconfigured to acquire a second left-eye image and a second right-eyeimage having parallax each of which is obtained by capturing thefluorescence from the observation target irradiated with the excitationlight; a superimposed image generation unit configured to superimposecorresponding pixels of the first left-eye image and the second left-eyeimage on each other to generate a left-eye fluorescence superimposedimage and superimpose corresponding pixels of the first right-eye imageand the second right-eye image on each other to generate a right-eyefluorescence superimposed image; and a display controller configured togenerate a display image from the first left-eye image, the firstright-eye image, the second left-eye image, the second right-eye image,the left-eye fluorescence superimposed image, and the right-eyefluorescence superimposed image, wherein the display image includes astereoscopically observable three-dimensional fluorescence superimposedimage generated from the left-eye fluorescence superimposed image andthe right-eye fluorescence superimposed image.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating a configuration of a medicalobservation system according to an embodiment;

FIG. 2 is a block diagram illustrating configurations of a camera headand a control device;

FIG. 3 is a flowchart illustrating an operation of the control device;

FIG. 4 is a view illustrating an operation of a memory controller in anormal observation mode;

FIG. 5 is a view illustrating the operation of the memory controller inthe normal observation mode;

FIG. 6 is a view illustrating an operation of the memory controller in afluorescence observation mode;

FIG. 7 is a view illustrating the operation of the memory controller inthe fluorescence observation mode;

FIG. 8 is a diagram illustrating a function of a display control unit;

FIG. 9 is a view illustrating the function of the display control unit;and

FIG. 10 is a view illustrating a modification of the embodiment.

DETAILED DESCRIPTION

Hereinafter, a mode (hereinafter, embodiment) for carrying out thepresent disclosure will be described with reference to the drawings.Incidentally, the present disclosure is not limited to the embodiment tobe described below. Further, the same parts are denoted by the samereference signs when the drawings are described.

Schematic Configuration of Medical Observation System

FIG. 1 is a diagram illustrating a configuration of a medicalobservation system 1 according to the present embodiment.

The medical observation system 1 is a system used in the medical fieldto capture (observe) the inside of a living body (observation target) asa subject. As illustrated in FIG. 1, the medical observation system 1includes an insertion section 2, a light source device 3, a light guide4, a camera head 5, a first transmission cable 6, a display device 7, asecond transmission cable 8, a control device 9, and a thirdtransmission cable 10.

The insertion section 2 is configured using a binocular relay type or amonocular pupil-division type scope (rigid endoscope).

Specifically, in the binocular relay type scope, two optical paths arearrayed in parallel in the scope. In addition, an optical system isarranged in each of the two optical paths. Further, the binocular relaytype scope takes and emits observation light for left and right eyeshaving parallax with each other in the two optical systems (see, forexample, JP H06-160731 A).

In addition, in the monocular pupil-division type scope, one opticalpath is provided in the scope. In addition, an optical system isarranged in the one optical path. Further, a pupil division unit thatdivides luminous flux in the pupil into two areas is provided at a pupilposition of the optical system. Further, the monocular pupil-divisiontype scope takes observation light in the optical system, divides theobservation light into observation light for left and right eyes havingparallax by the pupil division unit, and emits the divided observationlight (see, for example, JP H06-59199 A).

One end of the light guide 4 is connected with the light source device3, and supplies light that irradiates the inside of a living body to theone end of the light guide 4 under the control of the control device 9.As illustrated in FIG. 1, the light source device 3 includes a firstlight source 31 and a second light source 32.

The first light source 31 outputs (emits) light in a first wavelengthband. In the present embodiment, the first light source 31 is configuredusing an element that emits white light (light in the first wavelengthband). As the element that emits light, for example, a light emittingdiode (LED) or a laser diode (LD), which is a semiconductor element, maybe used.

The second light source 32 outputs (emits) excitation light in a secondwavelength band different from the first wavelength band. In the presentembodiment, the second light source 32 is configured using an elementthat emits near-infrared excitation light (excitation light in thesecond wavelength band). As the element that emits light, for example,an LED or an ED which is a semiconductor element may be used.

The near-infrared excitation light emitted by the second light source 32is excitation light that excites a fluorescent substance such asindocyanine green. In addition, when being excited with thenear-infrared excitation light, the fluorescent substance such asindocyanine green emits fluorescence having a center wavelength on thelonger wavelength side with respect to a center wavelength of awavelength band of the near-infrared excitation light. Incidentally, thewavelength band of the near-infrared excitation light and the wavelengthband of the fluorescence may be set so as to partially overlap with eachother, or may be set so as not to overlap at all.

Further, the first light source 31 is driven in a normal observationmode under the control of the control device 9 in the light sourcedevice 3 according to the present embodiment. That is, the light sourcedevice 3 emits normal light (white light) in the normal observationmode. The normal observation mode corresponds to a second observationmode according to the present disclosure. On the other hand, in afluorescence observation mode of the light source device 3, the firstlight source 31 is driven in a first period and the second light source32 is driven in a second period between the first and second periodsthat are alternately repeated under the control of the control device 9.That is, in the fluorescence observation mode, the light source device 3emits normal light (white light) in the first period and emitsnear-infrared excitation light in the second period. The fluorescenceobservation mode corresponds to a first observation mode according tothe present disclosure.

Incidentally, the light source device 3 is configured as a separate bodyfrom the control device 9 in the present embodiment, but may adopt aconfiguration of being provided inside the control device 9 withoutbeing limited thereto.

The one end of the light guide 4 is detachably connected with the lightsource device 3, and the other end thereof is detachably connected tothe insertion section 2. Further, the light guide 4 transmits light(normal light or near-infrared excitation light) supplied from the lightsource device 3 from one end to the other end and supplies the light tothe insertion section 2. When the inside of the living body isirradiated with the normal light (white light), the normal lightreflected in the living body is collected in the insertion section 2.Incidentally, for convenience of the description, the normal light asobservation light for left and right eyes, collected in the insertionsection 2 and emitted from the insertion section 2, is described asfirst left-eye and right-eye subject images hereinafter. In addition,when the inside of a living body is irradiated with near-infraredexcitation light, the near-infrared excitation light reflected in theliving body and fluorescence emitted from a fluorescent substance, suchas indocyanine green, accumulated in a lesion in the living body, as thefluorescent substance is excited, are collected in the insertion section2. Incidentally, for convenience of the description, the near-infraredexcitation light and fluorescence light as observation light for leftand right eyes, collected in the insertion section 2 and emitted fromthe insertion section 2, is described as second left-eye and right-eyesubject images hereinafter.

The camera head 5 corresponds to an imaging device according to thepresent disclosure. The camera head 5 is detachably connected to aproximal end (eyepiece 21 (FIG. 1)) of the insertion section 2. Further,the camera head 5 captures the first left-eye and right-eye subjectimages (normal light) and the second left-eye and right-eye subjectimages (near-infrared excitation light and fluorescence) emitted fromthe insertion section 2 and outputs an image signal obtained by thecapturing under the control of the control device 9.

Incidentally, a detailed configuration of the camera head 5 will bedescribed later.

One end of the first transmission cable 6 is detachably connected withthe control device 9 via a connector CN1 (FIG. 1), and the other endthereof is detachably connected with the camera head 5 via a connectorCN2 (FIG. 1). Further, the first transmission cable 6 transmits theimage signal and the like output from the camera head 5 to the controldevice 9, and transmits each of a control signal output from the controldevice 9, a synchronization signal, a clock signal, power, and the like,to the camera head 5.

Incidentally, in the transmission of the image signal. or the like fromthe camera head 5 to the control device 9 via the first transmissioncable 6, the image signal or the like may be transmitted as an opticalsignal or may be transmitted as an electrical signal. The same appliesto the transmission of the control signal, the synchronization signal,and the clock signal from the control device 9 to the camera head 5 viathe first transmission cable 6.

The display device 7 displays an image based on a video signal from thecontrol device 9.

One end of the second transmission cable 8 is detachably connected withthe display device 7, and the other end thereof is detachably connectedwith the control device 9. Further, the second transmission cable 8transmits the video signal processed by the control device 9 to thedisplay device 7.

The control device 9 corresponds to a medical image processing deviceaccording to the present disclosure. The control device 9 is configuredusing a central processing unit (CPU), a field-programmable gate array(FPGA), or the like, and performs the overall control of operations ofthe light source device 3, the camera head 5, and the display device 7.

Incidentally, a detailed configuration of the control device 9 will bedescribed later.

One end of the third transmission cable 10 is detachably connected withthe light source device 3, and the other end thereof is detachablyconnected with the control device 9. Further, the third transmissioncable 10 transmits the control signal from the control device 9 to thelight source device 3.

Configuration of Camera Head

Next, the configuration of the camera head 5 will be described.

FIG. 2 is a block diagram illustrating the configurations of the camerahead 5 and the control device 9.

Incidentally, FIG. 2 does not illustrate the connectors CN1 and CN2between the control device 9 and the camera head 5, and the firsttransmission cable 6, a connector between the control device 9 and thedisplay device 7, and the second transmission cable 8, and a connectorbetween the control device 9 and the light source device 3, and thethird transmission cable 10 for convenience of the description.

As illustrated in FIG. 2, the camera head 5 includes left-eye andright-eye imaging units 51 and 52 and a communication unit 53.

The left-eye imaging unit 51 captures the first left-eye subject image(normal light) and the second left-eye subject image (near-infraredexcitation light and fluorescence) emitted from the insertion section 2under the control of the control device 9. As illustrated in FIG. 2, theleft-eye imaging unit 51 includes a lens unit 511, an image sensor 513,and a signal processor 514.

The lens unit 511 is configured using one or a plurality of lenses, andforms the first left-eye subject image (normal light) and the secondleft-eye subject image (near-infrared excitation light and fluorescence)emitted from the insertion section 2 on an imaging surface of the imagesensor 513.

The image sensor 513 corresponds to a first image sensor according tothe present disclosure. The image sensor 513 is configured using acharge coupled device (CCD), a complementary metal oxide semiconductor(CMOS), or the like which optically receives light and converts thereceived light into an electrical signal (analog Signal).

Here, on the imaging surface (light-receiving surface) of the imagesensor 513, a color filter 513 a (FIG. 2) in which three filter groupsgrouped according to wavelength bands of transmitted light (R (red), G(green), and B (blue)) are arranged in a predetermined format (forexample, a Bayer array) is provided.

Specifically, the color filter 513 a has an R filter group that mainlytransmits light in the wavelength band of R, a B filter group thatmainly transmits light in the wavelength band of B, and a G filter groupthat mainly transmits light in the wavelength band of G.

Incidentally, the respective R, G, and B filter groups also transmitnear-infrared excitation light and fluorescence. Further, the imagesensor 513 has sensitivity not only to light in the wavelength bands ofR, G, and B, but also to light in wavelength bands of the near-infraredexcitation light and fluorescence.

Under the control of the control device 9, the image sensor 513 capturesthe first left-eye subject image (normal light) at a predetermined framerate in the normal observation mode, In addition, under the control ofthe control device 9, the image sensor 513 performs capturing in thefluorescence observation mode for each of the first and second periods,which are alternately repeated, in synchronization with the lightemission timing of the light source device 3.

Hereinafter, for convenience of the description, an image, generated bycapturing the first left-eye subject image (normal light) with the imagesensor 513, is described as a left-eye normal light image (correspondingto a first left-eye image according to the present disclosure). Inaddition, an image, generated by capturing the second left-eye subjectimage (near-infrared excitation light and fluorescence) with the imagesensor 513, is described as a left-eye fluorescence image (correspondingto a second left-eye image according to the present disclosure). Inaddition, the left-eye normal light image and the left-eye fluorescenceimage are collectively described as a left-eye captured image.

The signal processor 514 performs signal processing on the left-eyecaptured image (analog signal) generated by the image sensor 513.

The right-eye imaging unit 52 captures the first right-eye subject image(normal light) and the second right-eye subject image (near-infraredexcitation light and fluorescence) emitted from the insertion section 2under the control of the control device 9. As illustrated in FIG. 2, theright-eye imaging unit 52 includes a lens unit 521, an image sensor 523(color filter 523 a), and a signal processor 524, which are similar tothe lens unit 511, the image sensor 513 (including the color filter 513a), and the signal processor 514 in the left-eye imaging unit 51.

The image sensor 523 corresponds to a second image sensor according tothe present disclosure. Under the control of the control device 9, theimage sensor 523 captures the first right-eye subject image (normallight) at a predetermined frame rate in the normal observation mode. Inaddition, under the control of the control device 9, the image sensor523 performs capturing in the fluorescence observation mode for each ofthe first and second periods, which are alternately repeated, insynchronization with the light emission timing of the light sourcedevice 3.

Hereinafter, for convenience of the description, an image, generated bycapturing the first right-eye subject image (normal light) with theimage sensor 523, is described as a right-eye normal light image(corresponding to a first right-eye image according to the presentdisclosure). In addition, an image, generated by capturing the secondright-eye subject image (near-infrared excitation light andfluorescence) with the image sensor 523, is described as a right-eyefluorescence image (corresponding to a second right-eye image accordingto the present disclosure). In addition, the light-eye normal lightimage and the right-eye fluorescence image are collectively described asa right-eye captured image.

Further, the right-eye imaging unit 52 performs signal processing by thesignal processor 524 similarly to the left-eye imaging unit 51.

The communication unit 53 functions as a transmitter that transmits theleft-eye captured images in raster units, which are sequentially outputfrom the left-eye imaging unit 51 and the right-eye captured images inraster units, which are sequentially output from the right-eye imagingunit 52, to the control device 9 via the first transmission cable 6.

Configuration of Control Device

Next, the configuration of the control device 9 will be described withreference to FIG. 2.

As illustrated in FIG. 2, the control device 9 includes a communicationunit 91, first and second memories 92 and 93, an observation imagegeneration unit 94, a control unit 95, an input unit 96, an output unit97, and a storage unit 98.

The communication unit 91 functions as a receiver that receives theleft-eye and right-eye captured images in raster units which aresequentially output from the camera head 5 (communication unit 53) viathe first transmission cable 6. The communication unit 91 corresponds toa first captured image acquisition unit and a second captured imageacquisition unit according to the present disclosure.

The first memory 92 temporarily stores the left-eye and right-eyecaptured images sequentially output from the camera head 5(communication unit 53).

The second memory 93 temporarily stores the image processed by theobservation image generation unit 94.

Under the control of the control unit 95, the observation imagegeneration unit 94 processes the left-eye and right-eye captured imagesin raster units which are sequentially output from the camera head 5(communication unit 53) and received by the communication unit 91. Asillustrated in FIG. 2, the observation image generation unit 94 includesa memory controller 941, first to fourth image processors 942 to 945, asuperimposed image generation unit 946, and a display control unit 947.

The memory controller 941 controls writing of an image to the firstmemory 92 and reading of an image from the first memory 92 under thecontrol of the control unit 95. Incidentally, details of the function ofthe memory controller 941 will be described in “Operation of ControlDevice” which will be described later.

The first to fourth image processors 942 to 945 execute image processingin parallel on each of input images under the control of the controlunit 95.

Incidentally, the first to fourth image processors 942 to 945 have thesame configuration.

The superimposed image generation unit 946 operates only in thefluorescence observation mode under the control of the control unit 95.Further, the superimposed image generation unit 946 generates left-eyeand right-eye fluorescence superimposed images based on the images onwhich image processing has been executed by the first to fourth imageprocessors 942 to 945. Incidentally, details of the left-eye andright-eye fluorescence superimposed images will be described in“Operation of Control Device” which will be described later.

The display control unit 947 generates a display image from the imageson which the image processing has been executed by the first to fourthimage processors 942 to 945 and the left-eye and right-eye fluorescencesuperimposed images generated by the superimposed image generation unit946. Further, the display control unit 947 outputs a video signal fordisplay of the display image to the display device 7 via the secondtransmission cable 8. Incidentally, details of the function of thedisplay control unit 947 will be described in “Operation of ControlDevice” which will be described later.

The control unit 95 is configured using, for example, a CPU, an FPGA, orthe like, and outputs a control signal via the first to thirdtransmission cables 6, 8, and 10 to control the operations of the lightsource device 3, the camera head 5, and the display device 7 and controlthe entire operation of the control device 9. Incidentally details ofthe function of the control unit 95 will be described in “Operation ofControl device” which will be described later.

The input unit 96 is configured using an operation device such as amouse, a keyboard, and a touch panel, and receives a user operationperformed by a user such as a doctor. Further, the input unit 96 outputsan operation signal corresponding to the user operation to the controlunit 95.

The output unit 97 is configured using a speaker, a touch panel, or thelike, and outputs various types of information.

The storage unit 98 stores a program executed by the control unit 95,information necessary for processing of the control unit 95, and thelike.

Operation of Control Device

Next, the operation of the control device 9 described above will bedescribed.

FIG. 3 is a flowchart illustrating the operation of the control device9.

Incidentally, hereinafter, it is assumed that the image sensors 513 and523 are image sensors that generate left-eye and right-eye capturedimages having the number of pixels of 4K. In addition, it is assumedthat the maximum amount of data that may be processed by the first imageprocessor 942 is the amount of data of an image having the number ofpixels of full HD. The same applies to the other second to fourth imageprocessors 943 to 945.

First, the control unit 95 determines whether a current mode of thecontrol device 9 is the normal observation mode (Step S1).

Incidentally, the mode of the control device 9 is switched by thecontrol unit 95. Specifically, the control unit 95 switches the mode ofthe control device 9 to the normal observation mode or the fluorescenceobservation mode in response to a user operation on the input unit 96 bya user such as a doctor. That is, the control unit 95 has a function asa mode switching unit according to the present disclosure.

When determining that the current mode is the normal observation mode(Step S1: Yes), the control unit 95 drives the first light source 31(Step S2). That is, the inside of the living body is irradiated withnormal light (white light).

After Step S2, the control unit 95 causes the image sensors 513 and 523to capture the first left-eye and right-eye subject images (normallight) at a predetermined frame rate (Step S3). Further, the left-eyeand right-eye imaging units 51 and 52 sequentially output the left-eyeand right-eye normal light images each having the number of pixels of 4Kin raster units.

After Step S3, the memory controller 941 controls writing of an image tothe first memory 92 and reading of an image from the first memory 92(Step S4).

FIGS. 4 and 5 are views illustrating the operation of the memorycontroller 941 in the normal observation mode. Specifically, FIG. 4 is aview illustrating writing of the left-eye normal light image to thefirst memory 92. FIG. 5 is a view illustrating reading of the left-eyenormal light image from the first memory 92. Incidentally, FIGS. 4 and 5schematically illustrate a specific bank 921 among a plurality of banksin the first memory 92. The bank 921 corresponds to a fifth memory areaaccording to the present disclosure, and has a memory capacitycorresponding to the amount of data of an image having the number ofpixels of 4K in the present embodiment. In addition, the entire area inthe bank 921 is evenly divided into four areas of first to fourthdivided areas Ar1 to Ar4 in a rice crossed square shape in FIG. 5. Thatis, in the present embodiment, each of the first to fourth divided areasAr1 to Ar4 has a memory capacity corresponding to the amount of data ofan image having the number of pixels of full HD. The first to fourthdivided areas Ar1 to Ar4 correspond to divided areas according to thepresent disclosure.

Specifically, the memory controller 941 sequentially writes the left-eyenormal light images in raster units (the number of pixels: 4K),sequentially output from the left-eye imaging unit 51 and received bythe communication unit 91, to the bank 921 per line as indicated byarrows and broken lines in FIG. 4. Here, one arrow illustrated in FIG. 4indicates an image for one line in the left-eye normal light image (thenumber of pixels: 4K). Incidentally, the right-eye normal light imagesin raster units, sequentially output from the right-eye imaging unit. 52and received by the communication unit 91, are not used in theprocessing in Step S4 and the subsequent steps.

In addition, the memory controller 941 reads images written in the firstto fourth divided areas Ar1 to Ar4 sequentially from the first to fourthstorage positions P1 to P4 per line as indicated by arrows and dashedlines in FIG. 5 substantially at the same time as the timing of writingone frame of the left-eye normal light image (the number of pixels: 4K)to the fourth storage position P4 (FIG. 5).

Here, an image written in the first divided area Ar1 (hereinafter,referred to as a first divided image) is an image in a rectangular areaincluding an upper left corner position of the left-eye normal lightimage. Further, pixel data stored at the first storage position P1 ispixel data of a pixel at an upper left corner position in the firstdivided image. In addition, an image written in the second divided areaAr2 (hereinafter referred to as a second divided image) is an image in arectangular area including an upper right corner position of theleft-eye normal light image. Further, pixel data stored at the secondstorage position P2 is pixel data of a pixel at an upper left cornerposition in the second divided image. Further, an image written in thethird divided area Ar3 (hereinafter referred to as a third dividedimage) is an image in a rectangular area including a lower left cornerposition in the left-eye normal light image. Further, pixel data storedat the third storage position P3 is pixel data of a pixel at an upperleft corner position in the third divided image. In addition, an imagewritten in the fourth divided area Ar4 (hereinafter referred to as afourth divided image) is an image in a rectangular area including alower right corner position of the left-eye normal light image. Further,pixel data stored at the fourth storage position P4 is pixel data of apixel at an upper left corner position in the fourth divided image.

The first to fourth divided images described above are images obtainedby evenly dividing the left-eye normal light image having the number ofpixels of 4K into four images, and thus, each of the images has thenumber of pixels of full HD.

Further, the read first to fourth divided images (the number of pixels:full HD) are sequentially input to the first to fourth image processors942 to 945 per line. Incidentally, one arrow illustrated in FIG. 5indicates an image for one line in the first to fourth divided images(the number of pixels: full HD).

After Step 54, the first to fourth image processors 942 to 945 executeimage processing in parallel on the input first to fourth divided images(the number of pixels: full HD) (Step S5). After Step 55, the displaycontrol unit 947 generates a display image (left-eye normal light image(the number of pixels: 4K)) obtained by combining the first to fourthdivided images on which the first image processing has been executed(Step 56). Further, the display control unit 947 outputs a video signalfor display of the left-eye normal light image (the number of pixels:4K) to the display device 7 via the second transmission cable 8. As aresult, the display device 7 displays the left-eye normal light image(the number of pixels: 4K) based on the video signal.

Although the left-eye normal light image is used in Steps S4 to S6described above, the right-eye normal light image may be used withoutbeing limited thereto. Incidentally, the drive (capturing) of the imagesensor, that captures an image not used in Steps S4 to S6, between theimage sensors 513 and 523 may be stopped in Step S3.

When returning to Step Si and determining that the current mode is thefluorescence observation mode (Step S1: No), the control unit 95executes time-division drive of the first and second light sources 31and 32 (Step S7). Specifically, Step S7, the control unit 95 causes thefirst light source 31 to emit light in the first period between thefirst and second periods which are alternately repeated, and causes thesecond light source 32 to emit light in the second period based on thesynchronization signal.

After Step S7, the control unit 95 causes the light emission timings ofthe first and second light sources 31 and 32 to synchronize with eachother based on the synchronization signal, and causes the left-eye andright-eye imaging units 51 and 52 to capture the first left-eye andright-eye subject images and the second left-eye and right-eye subjectimages in the first and second periods, respectively (Steps S8 to S11).That is, in the first period (Step S8: Yes), in other words, when theinside of the living body is irradiated with normal light (white light),the image sensors 513 and 523 capture the first left-eye and right-eyesubject images (normal light) to generate the left-eye and right-eyenormal light images (Step S9). On the other hand, in the second period(Step S8: No), in other words, when the living body is irradiated withnear-infrared excitation light, the image sensors 513 and 523 capturethe second left-eye and right-eye subject images (near-infraredexcitation light and fluorescence) to generate the left-eye andright-eye fluorescence images (Step S10). In addition, the signalprocessors 514 and 524 execute reduction processing (Step 311). With thereduction processing, the left-eye and right-eye normal light images andthe left-eye and right-eye fluorescence images each having the number ofpixels of 4K are converted into the left-eye and right-eye normal lightimages and the left-eye and right-eye fluorescence images each havingthe number of pixels of full HD, respectively.

Further, the left-eye and right-eye imaging units 51 and 52 output theleft-eye and right-eye normal light images having the number of pixelsof full HD and obtained by capturing in the first period. In addition,the left-eye and right-eye imaging units 51 and 52 output the left-eyeand right-eye fluorescence images having the number of pixels of full HDand obtained by capturing in the second period.

After Step S11, the memory controller 941 controls writing of an imageto the first memory 92 and reading of an image from the first memory 92(Step S12).

FIGS. 6 and 7 are views illustrating the operation of the memorycontroller 941 in the fluorescence observation mode. Specifically, FIG.6 is a view illustrating writing of the left-eye and right-eye normallight images and the left-eye and right-eye fluorescence images to thefirst memory 92. FIG. 7 is a view illustrating reading of the left-eyeand right-eye normal light images and a left-eye and right-eyefluorescence images from the first memory 92. Incidentally, FIGS. 6 and7 schematically illustrate a specific bank 922 among the plurality ofbanks in the first memory 92. The bank 922 has the same memory capacityas the bank 921 (in the present embodiment, the memory capacitycorresponding to the amount of data of an image having the number ofpixels of 4K). In addition, the entire area in the bank 922 is evenlydivided into four areas of fifth to eighth divided areas Ar5 to Ar8 in arice crossed square shape in FIGS. 6 and 7. That is, each of the fifthto eighth divided areas Ar5 to Ar8 has the same memory capacity as eachof the first to fourth divided areas Ar1 to Ar4 (in the presentembodiment, the memory capacity corresponding to the amount of data ofan image having the number of pixels of full HD). Incidentally, thefifth to eighth divided areas Ar5 to Ar8 in the bank 922 correspond tofirst to fourth memory areas according to the present disclosure,respectively.

Specifically, the memory controller 941 sequentially writes the left-eyenormal light images in raster units (the number of pixels: full HD),sequentially output from the left-eye imaging unit 51 and received bythe communication unit 91, to the fifth divided area Ar5 in the bank 922per line as indicated by arrows and broken lines in FIG. 6. In addition,the memory controller 941 sequentially writes the right-eye normal lightimages in raster units (the number of pixels: full HD), sequentiallyoutput from the right-eye imaging unit 52 and received by thecommunication unit 91, to the sixth divided area Ar6 in the bank 922 perline at the same write timing as the left-eye normal light images.

Further, the memory controller 941 writes each one frame of the left-eyeand right-eye normal light images (the number of pixels: full HD) toeach of the fifth and sixth divided areas Ar5 and Ar6, and then,sequentially writes the left-eye fluorescence images in raster units(the number of pixels: full HD), sequentially output from the left-eyeimaging unit 51 and received by the communication unit 91, to theseventh divided area Ar7 in the bank 922 per line as indicated by arrowsand broken. lines in FIG, 6. In addition, the memory controller 941sequentially writes the right-eye fluorescence images in raster units(the number of pixels: full HD), sequentially output from the right-eyeimaging unit 52 and received by the communication unit 91, to the eighthdivided area Ar8 in the bank 922 per line at the same write timing asthe left-eye fluorescence images.

Incidentally, one arrow illustrated in FIG. 6 indicates an image for oneline in each of the left-eye and right-eye normal light images (thenumber of pixels: full HD) and the left-eye and right-eye fluorescenceimages (the number of pixels: full HD). In addition, in FIGS. 6 and 7,arrows at the same timing (write timing and read timing) in time arearrows having the same thickness. That is, arrows indicating the writetimings of the left-eye and right-eye normal light images (the number ofpixels: full HD) have the same thickness (thin arrows). On the otherhand, arrows indicating the write timings of the left-eye and right-eyefluorescence images (the number of pixels: full HD) have the samethickness, the thickness being different (thick arrow) from the arrowsindicating the write timings of the left-eye and right-eye normal lightimages.

In addition, the memory controller 941 sequentially reads the left-eyeand right-eye normal light images (the number of pixels: full HD) andthe left-eye and right-eye fluorescence images (the number of pixels:full HD) written in the fifth to eighth divided areas Ar5 to Ar8,respectively, from fifth to eighth storage positions P5 to P8 per linesubstantially at the same time as the timing of starting to write theleft-eye and right-eye fluorescence images (the number of pixels: fullHD) from the seventh and eighth storage positions P7 and P8 as indicatedby the arrows and dashed lines in FIG. 7. Incidentally, one arrowillustrated in FIG. 7 indicates an image for one line in each of theleft-eye and right-eye normal light images (the number of pixels: fullHD) and the left-eye and right-eye fluorescence images (the number ofpixels: full HD).

Here, pixel data stored at the fifth storage position P5 is pixel dataof a pixel at an upper left corner position in the left-eye normal lightimage (the number of pixels: full HD). In addition, pixel data stored atthe sixth storage position P6 is pixel data of a pixel at an upper leftcorner position in the right-eye normal light image (the number ofpixels: full HD). Further, pixel data stored at the seventh storageposition P7 is pixel data of a pixel at an upper left corner position inthe left-eye fluorescence image (the number of pixels: full HD). Inaddition, pixel data stored at the eighth storage position P8 is pixeldata of a pixel at an upper left corner position in the right-eyefluorescence image (the number of pixels: full HD).

Further, the read left-eye normal light images (the number of pixels:full HD) are sequentially input to the first image processor 942 perline. In addition, the read right-eye normal light images (the number ofpixels: full HD) are sequentially input to the second image processor943 per line. Further, the read left-eye fluorescence images (the numberof pixels: full HD) are sequentially input to the third image processor944 per line. In addition, the read right-eye fluorescence images (thenumber of pixels: full HD) are sequentially input to the fourth imageprocessor 945 per line.

After Step S12, the first to fourth image processors 942 to 945 executeimage processing in parallel on the input left-eye and right-eye normallight images (the number of pixels: full HD) and left-eye and right-eyefluorescence images (the number of pixels: full HD) (Step S13). Here,the first and second image processors 942 and 943 execute the firstimage processing on the input left-eye and right-eye normal light images(the number of pixels: full HD), respectively. On the other hand, thethird and fourth image processors 944 and 945 execute the second imageprocessing on the input left-eye and right-eye fluorescence images (thenumber of pixels: full HD), respectively.

After Step S13, the superimposed image generation unit 946 generatesleft-eye and right-eye fluorescence superimposed images SL and SR (seeFIG. 8) as illustrated below (Step S14).

Specifically, the superimposed image generation unit 946 superimposescorresponding pixels of the left-eye normal light image (the number ofpixels: full HD) on which the first image processing has been executedand the left-eye fluorescence image (the number of pixels: full HD) onwhich the second image processing has been executed on each other togenerate the left-eye fluorescence superimposed image SL (the number ofpixels: full HD). In addition, the superimposed image generation unit946 superimposes corresponding pixels of the right-eye normal lightimage (the number of pixels: full HD) on which the first imageprocessing has been executed and the right-eye fluorescence image (thenumber of pixels: full HD) on which the second image processing has beenexecuted on each other to generate the right-eye fluorescencesuperimposed image SR (the number of pixels: full HD).

Incidentally, examples of the superimposition may include a so-calledalpha blending process. The alpha blending process is a process ofsuperimposing pixel values (RGB values) of a first corresponding pixelcorresponding to a target pixel to be generated in a superimposed image(left-eye and right-eye fluorescence superimposed images) in abackground image (left-eye and right-eye normal light images) and RGBvalues of a specific pseudo color on each other at a ratio using analpha value for pixel values (RGB values) of the target pixel. The alphavalue is a value based on a fluorescent component of a secondcorresponding pixel corresponding to the target pixel in a fluorescenceimage (left-eye and right-eye fluorescence images).

After Step S14, the display control unit 947 generates a display imageas illustrated below (Step S15).

FIGS. 8 and 9 are views illustrating the function of the display controlunit 947. Specifically, in FIG. 8, reference signal “931” schematicallyindicates a specific bank among a plurality of banks in the secondmemory 93. In the present embodiment, the bank 931 has a memory capacitycorresponding to the amount of data of an image having the number ofpixels of 4K. In addition, the entire area in the bank 931 is evenlydivided into four areas of ninth to twelfth divided areas Ar9 to Ar12 ina rice crossed square shape in FIG. 8. That is, in the presentembodiment, each of the ninth to twelfth divided areas Ar9 to Ar12 has amemory capacity corresponding to the amount of data of an image havingthe number of pixels of full HD. In addition, in FIG. 8, the referencesign “IRL” indicates a left-eye fluorescence image (the number ofpixels: full HD) on which the second image processing has been executed.In addition, in FIG. 8, the reference sign “IRR” indicates a right-eyefluorescence image (the number of pixels: full HD) on which the secondimage processing has been executed. Incidentally, in the left-eye andright-eye fluorescence images IRL and IRR, an area that looks bright isan area where the intensity of a captured fluorescent component is high.In addition, in FIG. 8, reference signs “947 a” to “947 e” schematicallyindicate first to fifth enlargement processors constituting the displaycontrol unit 947. Further, in FIG. 8, reference sign “D1” schematicallyindicates a display image generated by the display control unit 947. Inaddition, in FIG. 8, reference sign “D2” schematically indicates adisplay image generated by the display control unit 947. In the presentembodiment, the display images D1 and D2 are three-dimensional imagesthat may be observed stereoscopically by a top-and-bottom method.Incidentally, the display image D1 corresponds to a three-dimensionalfluorescence superimposed image according to the present disclosure. Inaddition, the display image D2 corresponds to a second three-dimensionalimage according to the present disclosure. FIG. 9 is a view illustratingthe display images D1 and D2 displayed on the display device 7.

Specifically, the display control unit 947 selects an image to be usedto generate the display image from the left-eye and right-eye normallight images (the number of pixels: full HD) on which the first imageprocessing has been executed, the left-eye and right-eye fluorescenceimages IRL and IRR (the number of pixels: full HD), and the left-eye andright-eye fluorescence superimposed images SL and SR.

In the present embodiment, a display format in which the display imageD2 is displayed as a child image in a picture-in-picture with respect tothe display image D1 is set as illustrated in FIG. 9. Therefore, thedisplay control unit 947 selects the left-eye and right-eye fluorescenceimages IRL and IRR and the left-eye and right-eye fluorescencesuperimposed images St and SR from the left-eye and right-eye normallight images (the number of pixels: full HD) on which the first imageprocessing has been executed, the left-eye and right-eye fluorescenceimages IRL and IRR (the number of pixels: full HD), and the left-eye andright-eye fluorescence superimposed images SL and SR. Further, thedisplay control unit 947 sequentially writes the left-eye and right-eyefluorescence superimposed images SL and SR and the left-eye andright-eye fluorescence images IRL and IRR (the number of pixels: fullHD) to the ninth to twelfth divided areas Ar9 to Ar12, respectively, ofthe bank 931 in the second memory 93 per line at the same timing (FIG.8).

In addition, the first enlargement processor 947 a constituting thedisplay control unit 947 performs enlargement to double the number ofhorizontal pixels without increasing or decreasing the number ofvertical pixels while sequentially reading an image SL1 (FIG. 8) in theleft half of the left-eye fluorescence superimposed image SL written inthe ninth divided area Ar9 from the ninth divided area Ar9 per line.Incidentally, in FIG. 8, the left-eye fluorescence superimposed image SLis virtually divided into the image SL1 in the left half and an imageSL2 in the right half by a broken line.

Similarly, the second enlargement processor 947 b constituting thedisplay control unit 947 performs enlargement to double the number ofhorizontal pixels without increasing or decreasing the number ofvertical pixels while sequentially reading the image SL2 in the righthalf of the left-eye fluorescence superimposed image SL from the ninthdivided area Ar9 per line at the same read timing as the image SL1.

Further, the images processed by the first and second enlargementprocessors 947 a and 947 b are combined to generate a left-eyefluorescence superimposed image SL obtained by enlarging the left-eyefluorescence superimposed image SL to have the same number of verticalpixels and the doubled number of horizontal pixels.

In addition, the third enlargement processor 947 c constituting thedisplay control unit 947 performs enlargement to double the number ofhorizontal pixels without increasing or decreasing the number ofvertical pixels while sequentially reading an image SR1 (FIG. 8) in theleft half of the right-eye fluorescence superimposed image SR written inthe tenth divided area Ar10 from the tenth divided area Ar10 per line atthe same read timing as The images SL1 and SL2. Incidentally, in FIG. 8,the right-eye fluorescence superimposed image SR as virtually dividedinto the image SR1 in the left half and an image SR2 in the right halfby a broken line.

Similarly, the fourth enlargement processor 947d constituting thedisplay control unit 947 performs enlargement to double the number ofhorizontal pixels without increasing or decreasing the number ofvertical pixels while sequentially reading the image SR2 in the righthalf of the right-eye fluorescence superimposed image SR from the tenthdivided area Ar10 per line at the same read timing as the images SL1,SL2, and SR1.

Further, the images processed by the third and fourth enlargementprocessors 947 c and 947 d are combined to generate a right-eyefluorescence superimposed image SR′ obtained by enlarging the right-eyefluorescence superimposed image SR to have the same number of verticalpixels and the doubled number of horizontal pixels.

That is, the display control unit 947 generates the display image D1(the number of pixels: 4K) that is obtained by combing the left-eye andright-eye fluorescence superimposed images SL′ and SR′ and may beobserved stereoscopically by the top-and-bottom method. Further, thedisplay control unit 947 outputs a first video signal for display of thedisplay image D1 (the number of pixels: 4K) to the display device 7 viathe second transmission cable 8. As a result, the display device 7displays the display image D1 (the number of pixels: 4K) based on thefirst video signal as a parent image in the picture-in.-picture asillustrated in FIG. 9.

In addition, the fifth enlargement processor 947 e constituting thedisplay control unit 947 reduces the number of vertical pixels to halfwithout increasing or decreasing the number of horizontal pixels whilesequentially reading the left-eye fluorescence image IRL written in theeleventh divided area Ar11 from the eleventh divided area Ar11 per lineat the same read timing as the images SL1, SL2, SR1, and SR2. Inaddition, after reading one frame of the left-eye fluorescence imageIRL, the fifth enlargement processor 947 e reduces the number ofvertical pixels to half without increasing or decreasing the number ofhorizontal pixels while sequentially reading the right-eye fluorescenceimage IRR written in the twelfth divided area Ar12 sequentially from thetwelfth divided area Ar12 per line.

Further, by the processing of the fifth enlargement processor 947 e, aleft-eye fluorescence image IRL′, which has the same number ofhorizontal pixels and the number of vertical pixels reduced by half ascompared with the left-eye fluorescence image IRL, and a right-eyefluorescence image IRR′, which has the same number of horizontal pixelsand the number of vertical pixels reduced by half as compared with theright-eye fluorescence image IRR, are generated.

That is, the display control unit 947 generates the display image D2(the number of pixels: full HD) that is obtained by combining theleft-eye and right-eye fluorescence images IRL′ and IRR′ and may beobserved stereoscopically by the top-and-bottom method. Further, thedisplay control unit 947 outputs a second video signal for display ofthe display image D2 (the number of pixels: full HD) to the displaydevice 7 via the second transmission cable 8. As a result, asillustrated in FIG. 9, the display device 7 displays the display imageD2 (the number of pixels: full HD) based on the second video signal asthe child image in the picture-in-picture.

Incidentally, the above-described display format may be changed insetting according to a user operation on the input unit 96 by a usersuch as a doctor. Examples of the display format that may be set mayinclude the following first to fourth display formats in addition to theabove-described display format in which the picture-in-picture isdisplayed with the display image D1 as the parent image and the displayimage D2 as the child image.

The first display format is a display format in which apicture-in-picture is displayed with the display image D1 as a parentimage and a first three-dimensional image as a child image. The firstthree-dimensional image is a stereoscopically observablethree-dimensional image generated from left-eye and right-eye normalimages (the number of pixels: full HD) on which the first imageprocessing has been executed.

The second display format is a display format in which apicture-in-picture is displayed with the display image D1 as a parentimage and the display image D2 and the first three-dimensional image aschild images.

The third display format is a display format that displays only thedisplay image D1.

The fourth display format is a display format in which apicture-in-picture is displayed with the display image D1 as a parentimage and any one of left-eye and right-eye normal light images andleft-eye and right-eye fluorescence images as child images.

Incidentally, the above description has been given using thepicture-in-picture as the display format, but the display format is notlimited to the picture-in-picture. If a format (composite image displayformat) that displays (combines) a plurality of pieces of differentimage information on one screen is adopted, the equivalent functions andeffects are achieved regardless of the display format of thepicture-in-picture.

According to the present embodiment described above, the followingeffects are achieved.

The control device 9 according to the present embodiment generates thestereoscopically observable display image D1 obtained by combining theleft-eye and right-eye fluorescence superimposed images SL′ and SR′.Therefore, a doctor or the like may easily excise or suture a lesionwhile confirming the display image D1 having the sense of depthdisplayed on the display device 7 Therefore, the convenience may beimproved with the control device 9 according to the present embodiment.

In addition, the control device 9 according to the present embodimentgenerates the stereoscopically observable display image D2 obtained bycombining the left-eye and right-eye fluorescence images IRL′ and IRR′to use the display image D2 as the child image displayed in thepicture-in-picture with respect to the display image D1. Therefore, adoctor or the like may confirm both the display images D1 and D2displayed on the display device 7, and more easily grasp a position of alesion and more easily excise or suture the lesion.

In addition, the control device 9 according to the present embodimentexecutes the image processing in parallel on the first to fourth dividedimages using the first to fourth image processors 942 to 945 in thenormal observation mode. On the other hand, the control device 9executes the image processing in parallel on the left-eye and right-eyenormal light images and the left-eye and right-eye fluorescence imagesusing the first to fourth image processors 942 to 945 in thefluorescence observation mode.

That is, the image processor that executes the image processing inparallel on the first to fourth divided images and the image processorthat executes the image processing in parallel on the left-eye andright-eye normal light images and the left-eye and right-eyefluorescence images may be configured as the common image processor.Therefore, it is possible to generate an image suitable for observationwithout increasing a circuit scale.

Other Embodiments

The modes for carrying out the present disclosure have been describedhereinbefore. However, the present disclosure is not limited only to theembodiments described above.

Although the three-dimensional images stereoscopically observable by thetop-and-bottom method are illustrated as the display images D1 and D2 inthe above-described embodiment, the present disclosure is not limitedthereto, and a display image that is a three-dimensional image that maybe stereoscopically observed by a side-by-side or line-by-line methodmay be generated.

In the above-described embodiment, the left-eye and right-eye normallight images used when Generating the left-eye and right-eyefluorescence superimposed images may be the following images by thefirst image processing executed by the first and second image processors942 and 943.

That is, the first and second image processors 942 and 943 remove rvalues in the left-eye and right-eye normal light images to leave only gvalues and b values for conversion into images that do not include R inpixel values. Further, the superimposed image generation unit 946superimposes corresponding pixels of the converted images and theleft-eye and right-eye fluorescence images on each other to generateleft-eye and right-eye fluorescence superimposed images.

The number of each of the first to fourth image processors according tothe present disclosure is one in the above-described embodiment, but maybe two or more without being limited thereto. For example, in a casewhere a left-eye normal light image or a right-eye normal light imagehaving the number of pixels of 8K is processed in the normal observationmode, it is necessary to provide a total of sixteen image processors inwhich the number of each of the first to fourth image processorsaccording to the present disclosure is four when assuming the use of theimage processor in which the maximum amount of data that may beprocessed is the amount of data of a full HD image as in theabove-described embodiment.

The image sensors 513 and 523 are configured using the image sensor thatgenerates the image having the number of pixels of 4K in theabove-described embodiment, but may be configured using an image sensorthat generates an image having another number of pixels without beinglimited thereto.

In the above-described embodiment, a configuration in which only thefluorescence observation mode is provided without providing the normalobservation mode may be adopted.

FIG. 10 is a view illustrating a modification of the embodiment.Incidentally, in the modification illustrated in FIG. 10, the left-eyeimaging unit 51 is described as a left-eye imaging unit 51A, and theright-eye imaging unit 52 is described as a right-eye imaging unit 52Afor convenience of the description.

Although the light in the first wavelength band and the excitation lightin the second wavelength band are emitted in a time-division manner inthe fluorescence observation mode in the above-described embodiment, thepresent disclosure is not limited thereto. For example, on a side wherethe light in the first wavelength band and the excitation light in thesecond wavelength band are emitted and captured at the same time, thelight in the first wavelength band and the excitation light andfluorescence in the second wavelength band are separated by a filter 512(522) provided in an optical path of the left-eye imaging unit 51A(right-eye imaging unit 52A) as illustrated in FIG. 10. The filter 512corresponds to a first filter according to the present disclosure. Thefilter 522 corresponds to a second filter according to the presentdisclosure. Further, two image sensors 5131 (5231) and 5132 (5232) areprovided in the left-eye imaging unit 51A (right-eye imaging unit 52A).The image sensor 5131 is an image sensor having sensitivity to the lightin the first wavelength band, and corresponds to the first image sensoraccording to the present disclosure. The image sensor 5231 is an imagesensor having sensitivity to the light in the first wavelength band, andcorresponds to a second image sensor according to the presentdisclosure. The image sensor 5132 is an image sensor having sensitivityto the light in the second wavelength band, and corresponds to a thirdimage sensor according to the present disclosure. The image sensor 5232is an image sensor having sensitivity to the light in the secondwavelength band, and corresponds to a fourth image sensor according tothe present disclosure. That is, a configuration may be adopted in whichthe two image sensors 5131 (5231) and 5132 (5232) are provided in theleft-eye imaging unit 51A (right-eye imaging unit 52A) and these twoimage sensors 5131 (5231) and 5132 (5232) capture optical images in thefirst and second wavelength bands, respectively.

Further, as described above, the filter groups of R, G, and Bconstituting the color filters 513 a and 523 a of the image sensors 513and 523 transmit not only the light in the wavelength bands of R, G, andB but also the near-infrared excitation light and fluorescence. Further,the image sensors 513 and 523 have sensitivity not only to the light inthe respective wavelength bands of R, G, and B, but also to the light inwavelength bands of the near-infrared excitation light and fluorescence.Utilizing this characteristic, it may be configured such that imageinformation obtained by the image sensors 513 and 523 is separated bythe observation image generation unit 94 into optical image informationin the first wavelength band and optical image information in the secondwavelength band to generate left-eye and right-eye fluorescencesuperimposed images.

In addition, it may be configured such that a color filter havingsensitivity mainly to light in wavelength bands of near-infraredexcitation light and fluorescence is added to the color filters 513 aand 523 a of the imaging image sensors 513 and 523 described above,image information obtained by the image sensors 513 and 523 is separatedby the observation image generation unit 94 to acquire optical imageinformation in the first wavelength band and optical image informationin the second wavelength band, and left-eye and right-eye fluorescencesuperimposed images are generated.

Although the light in the first wavelength band is white light and theexcitation light in the second wavelength band is near-infraredexcitation light in the above-described embodiment, the presentdisclosure is not limited thereto. As the first and second light sources31 and 32, other configurations may be adopted as long as the firstlight source 31 emits light in a first wavelength band and the secondlight source 32 emits light in a second wavelength band different fromthe first wavelength band. At this time, the first and second wavelengthbands may be partially overlapping bands or may be bands which do notoverlap at all.

Meanwhile, there is known photodynamic diagnosis (PDD), which is one ofcancer diagnostic methods for detecting cancer cells.

In the photodynamic diagnosis, a photosensitive substance such as5-aminolevulinic acid (hereinafter referred to as 5-ALA) is used. The5-ALA is a natural amino acid originally contained in a living body ofan animal or a plant. This 5-ALA is taken into a cell afteradministration into the body and biosynthesized into protoporphyrin in amitochondrion. Further, the protoporphyrin is excessively accumulated inthe cancer cell. In addition, the protoporphyrin excessively accumulatedin the cancer cell has photoactivity. Therefore, when excited withexcitation light (for example, blue visible light in a wavelength bandof 375 nm to 445 nm), the protoporphyrin emits fluorescence (forexample, red fluorescence in a wavelength band of 600 nm to 700 nm). Thecancer diagnostic method in which a light-sensitive substance is used tocause the cancer cell to emit the fluorescence in this mariner is calledphotodynamic diagnosis.

Further, in the above-described embodiment, the first light source 31may be configured using an LED that emits white light, and the secondlight source 32 may be configured using a semiconductor laser that emitsexcitation light that excites protoporphyrin (for example, blue visiblelight in the wavelength band of 375 nm to 445 nm). Even in the case ofadopting such a configuration, the same effects as those of theabove-described embodiment may be obtained.

Although the first and second periods are set to alternately repeat inthe fluorescence observation mode in the embodiment described above, thepresent disclosure is not limited thereto, and may be configured suchthat at least one of the first and second periods is continuous and afrequency ratio of the first and second periods is a ratio other than1:1.

Although the medical image processing device according to the presentdisclosure is mounted on the medical observation system 1 in which theinsertion section 2 is configured using the rigid endoscope in theabove-described embodiment, the present disclosure is not limitedthereto. For example, the medical image processing device according tothe present disclosure may be mounted on a medical observation system inwhich the insertion section 2 is configured using a flexible endoscope.In addition, the medical image processing device according to thepresent disclosure may be mounted on a medical observation system suchas a surgical microscope (see, for example, JP 2016-42981 A) thatmagnifies and observes a predetermined visual field area in a subject(inside a living body) or on a surface of the subject (living bodysurface).

In the above-described embodiment, some configurations of the camerahead 5 and some configurations or the control device 9 may be providedin the connector CN1 or the connector CN2, for example.

With the medical image processing device and the medical observationsystem according to the present disclosure, the convenience may beimproved.

Although the disclosure has been described with respect to specificembodiments for a complete and clear disclosure, the appended claims arenot to be thus limited but are to be construed as embodying allmodifications and alternative constructions that may occur to oneskilled in the art that fairly fall within the basic teaching herein setforth.

What is claimed is:
 1. A medical image processing device comprising: afirst captured image acquisition unit configured to acquire a firstleft-eye image and a first right-eye image having parallax each of whichis obtained by capturing light from an observation target irradiatedwith light in a first wavelength band, the observation target emittingfluorescence when irradiated with excitation light in a secondwavelength band different from the first wavelength band; a secondcaptured image acquisition unit configured to acquire a second left-eyeimage and a second right-eye image having parallax each of which isobtained by capturing the fluorescence from the observation target15irradiated with the excitation light; a superimposed image generationunit configured to superimpose corresponding pixels of the firstleft-eye image and the second left-eye image on each other to generate aleft-eye fluorescence superimposed image and superimpose correspondingpixels of the first right-eye image and the second right-eye image oneach other to generate a right-eye fluorescence superimposed image; anda display controller configured to generate a display image from thefirst left-eye image, the first right-eye image, the second left-eyeimage, the second right-eye image, the left-eye fluorescencesuperimposed image, and the right-eye fluorescence superimposed image,wherein the display image includes a stereoscopically observablethree-dimensional fluorescence superimposed image generated from theleft-eye fluorescence superimposed image and the right-eye fluorescencesuperimposed image.
 2. The medical image processing device according toclaim. 1, wherein each of the first left-eye image and the second lefteye image is generated by a left-eye imaging unit including a firstimage sensor, and each of the first right-eye image and the secondright-eye image is generated by a right-eye imaging unit including asecond image sensor.
 3. The medical image processing device according toclaim 1, wherein the first left-eye image and the first right-eye imageare generated based on the light in the first wavelength band, and thesecond left-eye image and the second right-eye image are generated basedon the light in the second wavelength band.
 4. The medical imageprocessing device according to claim 1, wherein the light in the firstwavelength band is light emitted in a first period between the firstperiod and a second period which are alternately repeated, and the lightin the second wavelength band is light emitted in the second period. 5.The medical image processing device according to claim 2, wherein eachof the first image sensor and the second image sensor includes a filterconfigured to transmit the light in the second wavelength band.
 6. Themedical image processing device according to claim 2, wherein theleft-eye imaging unit includes: a first filter configured to separatethe light in the first wavelength band and the light in the secondwavelength band; the first image sensor configured to receive the lightin the first wavelength band separated by the first filter; and a thirdimage sensor configured to receive the light in the second wavelengthband separated by the first filter, and the right-eye imaging unitincludes: a second filter configured to separate the light in the firstwavelength band and the light in the second wavelength band; the secondimage sensor configured to receive the light in the first wavelengthband separated by the second filter; and a fourth image sensorconfigured to receive the light in the second wavelength band separatedby the second filter.
 7. The medical image processing device accordingto claim 1, wherein the display image includes the three-dimensionalfluorescence superimposed image, and an image based on the firstleft-eye image, the first right-eye image, the second left-eye image,and the second right-eye image, the image displayed as a composite imagewith respect to the three-dimensional fluorescence superimposed image.8. The medical image processing device according to claim 7, wherein thecomposite image includes a child image displayed in a picture-in-picturewith respect to the three-dimensional fluorescence superimposed image.9. The medical image processing device according to claim 7, wherein thecomposite image is at least one of: a stereoscopically observable firstthree-dimensional image generated from the first left-eye image and thefirst right-eye image; and a stereoscopically observable secondthree-dimensional image generated from the second left-eye image and thesecond right-eye image.
 10. The medical image processing deviceaccording to claim 1, further comprising: a first image processorconfigured to execute image processing on the first left-eye image; asecond image processor configured to execute image processing on thefirst right-eye image; a third image processor configured to executeimage processing on the second left-eye image; a fourth image processorconfigured to execute image processing on the second right-eye image; amemory configured to temporarily store an image; and a memory controllerconfigured to write the first left-eye image, the first right-eye image,the second left-eye image, and the second right-eye image to a firstmemory area, a second memory area, a third memory area, and a fourthmemory area, respectively, which have an identical memory capacity inthe memory, read the first left-eye image, the first right-eye image,the second left-eye image, and the second right-eye image from the firstmemory area, the second memory area, the third memory area, and thefourth memory area, respectively, and output the read first left-eyeimage, first right-eye image, second left-eye image, and secondright-eye image to the first image processor, the second imageprocessor, the third image processor, and the fourth image processor,respectively, wherein the superimposed image generation unit isconfigured to generate the left-eye fluorescence superimposed image andthe right-eye fluorescence superimposed image based on the firstleft-eye image, the first right-eye image, the second left-eye image,and the second right-eye image on which the image processing has beenexecuted by the first image processor, the second image processor, thethird image processor, and the fourth image processor.
 11. The medicalimage processing device according to claim 10, further comprising a modeswitching unit configured to switch between a first observation mode ora second observation mode, wherein the memory controller is configuredto write the first left-eye image, the first right-eye image, the secondleft-eye image, and the second right-eye image to the first memory area,the second memory area, the third memory area, and the fourth memoryarea, respectively, read the first left-eye image, the first right-eyeimage, the second left-eye image, and the second right-eye image fromthe first memory area, the second memory area, and the third memoryarea, and the fourth memory area, respectively, and output the readfirst left-eye image, the first right-eye image, the second left-eyeimage, and the second right-eye image to the first image processor, thesecond image processor, the third image processor, and the fourth imageprocessor, respectively, in the first observation mode, and write anyone of the first left-eye image and the first right-eye image to a fifthmemory area in the memory, reads four divided images obtained bydividing the one image from four divided areas in the fifth memory areain which the four divided images have been written, and outputs the readfour divided images to the first image processor, the second imageprocessor, the third image processor, and the fourth image processor, inthe second observation mode, and the superimposed image generation unitis configured to generate the left-eye fluorescence superimposed imageand the right-eye fluorescence superimposed image in the firstobservation mode.
 12. The medical image processing device according toclaim 11, wherein the first image processor and the second imageprocessor are configured to execute first image processing on each ofthe first left-eye image and the first right-eye image in the firstobservation mode, and execute the first image processing on each of thedivided images in the second observation mode, and the third imageprocessor and the fourth image processor are configured to executesecond image processing different from the first image processing oneach of the second left-eye image and the second right-eye image in thefirst observation mode, and execute the first image processing on eachof the divided images in the second observation mode.
 13. The medicalimage processing device according to claim 11, wherein in the memory,the one image of which a total number of pixels is a first number ofpixels is written to the fifth memory area in the second observationmode, and the first left-eye image, the first right-eye image, thesecond left-eye image, and the second right-eye image on which reductionprocessing has been performed and each of which has a total number ofpixels being a second number of pixels, the second number of pixelsbeing equal to or smaller than 1/4 of the first number of pixels, arewritten to the first memory area, the second memory area, the thirdmemory area, and the fourth memory area, respectively, in the firstobservation mode.
 14. A medical observation system comprising: a lightsource device configured to emit light in a first wavelength band andexcitation light in a second wavelength band different from the firstwavelength band; an imaging device configured to capture light from anobservation target irradiated with the light in the first wavelengthband to generate a first left-eye image and the first right-eye imagehaving parallax, the observation target emitting fluorescence whenirradiated with excitation light, and capture the fluorescence from theobservation target irradiated with the excitation light to generate asecond left-eye image and a second right-eye image having parallax; andthe medical image processing device according to claim 1 configured toprocess the first left-eye image, the first right-eye image, the secondleft-eye image, and the second right-eye image.